The present invention relates to computer aided software engineering and more particularly to a dynamic and semi-structured repository for reverse engineering software systems.
Software systems are business assets, and typically critical assets. Existing software systems have become increasingly larger and more complex with lifetimes that are much longer than planned. Systems, as a whole, continually change. These changes, in turn, must propagate to all subsystems including software systems. Changes in a software system can be due to changes in the requirement specification, for example a change in government regulations, or as a result of changes in the operating environment, for example a change in the operating system. Other common changes can be corrective and in response to the detection of xe2x80x9cbugsxe2x80x9d. Software systems are called legacy systems when they are mature, large in size, complex, and their documentation is limited. Generally such systems are business assets that cannot be replaced or retired easily. The existence of a large number of legacy systems has generated interest in building computer software tools that help understand the architecture and functionality of these systems.
The issue of change in software systems is often addressed under the title of maintenance or evolution. It is well known that software maintenance is both critical and time consuming. Reports of software maintenance research suggest that a large portion of the software life cycle (50% to 90%) is invested in maintenance.
A primary difficulty in maintaining software is coming to an understanding of the underlying source code. Software maintenance engineers are generally different than original designers, who are in most cases no longer available. Furthermore, many companies assign their newly hired programmers to the task of maintenance of software systems. The main source of understanding, in such cases, is the internal documentation which was prepared for the software. Software documentation is generally incomplete, outdated, or simply non-existent. Under the competitive pressures of today""s software industry, documenting programs is considered secondary and often treated as an afterthought. In cases where documentation is available and up-to-date, it still may provide little help since it is often written text, augmented with sketches that provide only one view of the software system. Thus, a large portion of maintenance time is spent on understanding the software system by investigating the actual source code and talking to programmers with more expertise (i.e., mentoring).
The problem of maintenance is exacerbated by the short turn around time of software professionals: on average two to three years. Programmers either change projects, take promotions, or move for employment elsewhere. The combination of program understanding difficulties and short turnaround time of programmers has resulted in higher costs of training and, in turn, higher costs for maintaining the software systems.
The proposed solution to the problems of maintaining existing software systems involves the use of reverse engineering techniques.
Reverse engineering is the process of understanding how an existing software system works. This process involves extracting data about the software system and creating higher-level abstractions that simplify understanding of the system. In simple terms, reverse engineering involves mapping of the existing software artifact to a design specification and from there to a system specification. The mapping from specification to an implementation is one to many. Based on the knowledge of programming and the application domain, high-level abstractions are constructed. These abstractions represent different views of the system that capture only some essential properties of the software system. Some aspects of reverse engineering, such as data extraction, can be automated, while others cannot. It will be appreciated that reverse engineering even with the development of various reverse engineering tools remains a semi-automated process, which requires the involvement of human programmers.
Known reverse engineering tools provide support in the processes of extraction, analysis, and understanding of complex software systems during maintenance and evolution. Typically these tool sets include processes for parsing source code, grouping and filtering of the parsed code, representing derived information in textual and/or graphic forms, and browsing.
The Refine/C(trademark) utility is a known extensible workbench for reverse engineering of existing C programs. The Refine/C(trademark) utility provides a C parser and a lexical analyzer and APIs for building customized analysis tools. The Rigi(trademark) program, which arose out of research at the University of Victoria, is another reverse engineering tool which is based on a graphical editor called rigiedit. The Rigi(trademark) tool provides support for partial parsing of programs written in C, C++, PL/I, and COBOL. The Rigi(trademark) tool also provides browsing and editing facilities for entities generated by parsing the source code.
The Imagix 4D(trademark) program is a reverse engineering tool for C and C++ programs from the Imagix Corporation. The Imagix 4D(trademark) program provides parsing facilities for C and C++ programs as well as views of the program at different levels of abstractions. These views are presented in a 3D format that allows the user to focus on those of immediate interest.
Also of note are tools that were developed for the purpose of architectural discovery. An example of such a tool is the Dali(trademark) program, which was developed by Software Engineering Institute. The Dali(trademark) program is a combination of various tools. It uses existing parsers for extracting information, a relational database for storing the extracted information, and Rigi(trademark) for viewing and editing the extracted information. Each of the tools mentioned here has been applied to moderate sized software systems with some success.
In view of the foregoing, it will be appreciated that reverse engineering inherently is an ill-defined problem due to the difference in levels of expertise of the programmers. As a result, the process of reverse engineering has been characterized as exploratory. Many approaches have been put forward for reverse engineering, each with a varying degree of success in industrial settings. These approaches typically use one or more cognitive models of program understanding and support the reverse engineering process by providing a number of tools for extraction, analysis, visualization, and manual editing and browsing. In other words, all these approaches are tool-based. Unfortunately, reverse engineering is a time consuming endeavour. It typically takes months before a model of the software system being investigated is constructed, and these models are often inadequate in a number of ways. First, owners of software systems generally do not have an ability to generate and investigate models of the existing software system. The models are mostly constructed by using specific tools and the creation of new models requires the construction of new tools or the modification of existing ones. Second, searching capabilities are limited. In some cases, relational databases are used for intermediate storage of extracted information. Relational databases provide efficient facilities for querying databases. However, these relational databases are generally not suitable for interactive query of software repositories. As a result, relational databases are usually used during early stages for xe2x80x9coff-linexe2x80x9d querying. Once the higher-level abstractions are constructed, the relational database is no longer used. Thus, we need a combination of database and knowledge base technologies. Third, most conventional reverse engineering tools lack history and replay mechanisms. The process or reverse engineering, in other words, is once again in the minds of those involved in the initial process. Fourth, reverse engineering tools are centered around parsing engines. These tools rarely provide partial or incremental parsing, and as a result they are generally weak in incremental model building and reasoning in the presence of incomplete and unresolved references. Fifth, the tool-centric views of reverse engineering approaches do not provide sufficient support for integrating external tools.
The increase in size and complexity of software systems combined with their long lifetime has shifted the focus from design to maintenance and evolution. A key issue in maintaining legacy software systems is to understand them. Understanding large software systems involves abstracting away nonessential details and focussing on what is needed for the task at hand. For example, if changes in a software specification require integrating a new subsystem, it may only be necessary to understand the interfaces for the software system as opposed to details of the entire system.
Current reverse engineering technologies focus mainly on ad hoc tools that are designed for specific tasks: construction of data-flow graphs, control-flow graphs, call graphs, etc. The information captured by these tools, and the knowledge invested in construction of them are not readily usable by other systems. Further, software systems, as well as the personnel involved in maintaining them, continually change. There must be a way to understand software systems that can respond to continuous change. The challenge is to build tools that support knowledge interchange and work in dynamic environments.
Accordingly, in view of the shortcomings associated with existing reverse engineering techniques there still remains a need in the art. The present invention provides a repository-based system for reverse engineering software, particularly legacy software systems. The repository-based system according to the invention provides mechanisms for incremental building of models of the software system, filtering, grouping, manual manipulation, navigation and querying of software systems as well as history and replay facilities for recreation of models in lieu of changes to the software system. Advantageously, the repository mechanisms are useable with different programming languages and provide the capability for integrating external software tools.
The present invention provides a repository-based system for reverse engineering software, and is particularly suited for legacy software systems.
The repository-based reverse engineering system according to the present invention combines database and knowledge base technologies which enable the user to capture historical information that is required for dealing with changes in the software system. The repository-based reverse engineering system provides the capability to store extracted data without enforcing a predetermined model, to make incremental updates to the extracted data, to capture knowledge about programming and the software system, to provide analysis tools for finding structure in the extracted data, to interface to external tools such as visualization tools, and to support export and import of information. The repository-based reverse engineering system complements existing technologies for data extraction, analysis, and visualization by providing uniform access to the data extracted from the software system and abstract models build from this data. Furthermore, historical data, in the form of records of operations of the data, would provide insight into the data extraction, generation of models, and the results of previous analyses performed on the software system.
The repository-based reverse engineering system features a software information infrastructure for managing the reverse engineering of computer programs in software systems. The repository-based reverse engineering system provides the capability to integrate heterogeneous information sources, to integrate external visualization and analytical tools, and to integrate search and browsing utilities. Advantageously, the storage of semi-structured information in the repository supports highly-evolving data management.
The repository-based reverse engineering system includes the following features: a mechanism for flexible graph-based storage of information; and mechanisms for creating multiple levels of information abstractions. The repository-based system includes scalability support through connection to a conventional relational database such as DB2(trademark). The repository-based system also features a flexible search and browse facility for heterogeneous data sources based on pattern matching, structural searching, fuzzy and incomplete searching, and similarity-based searching.
In one embodiment, the repository-based reverse engineering system according to the present invention is implemented in the known Java(trademark) programming language. In another embodiment, XML is the specification language for the repository-based reverse engineering system.
In a first aspect, the present invention provides a system for reverse engineering a computer program, the system includes: (a) means for extracting information related to source code in the computer program; (b) means for storing the extracted information in a semi-structured form; (c) means for creating a plurality of representations of the semi-structured information; and (d) means for updating the semi-structured information.
In another aspect, the present invention provides a method for reverse engineering a computer program, the method comprising the steps of: (a) extracting information related to source code in the computer program; (b) storing the extracted information in a semi-structured form; (c) creating a plurality of representations of the semi-structured information, wherein the representations provide means for analyzing the source code.
In yet another aspect, the present invention provides a Data storage media recorded with a computer program which, in combination with a general purpose computer configured for a system for reverse engineering a legacy computer program and said computer being equipped to read into memory and execute program data from the data storage media, constituting a method in accordance with the present invention.